Model Transfer for Markov Decision Tasks via Parameter Matching

Model transfer refers to the process of transferring information from a model that was previously identified for one task (source) to a new task (target). For decision tasks in unknown Markov environments, we profit through model transfer by using information from related tasks, e.g. transition knowledge and solution (policy) knowledge, to quickly determine an appropriate model of the new task environment. A difficulty with such transfer is typically the non-linear and indirect relationship between the available source knowledge and the target’s working prior model of the unknown environment, provided through a complex multidimensional transfer function. In this paper, we take a Bayesian view and present a probability perturbation method that conditions the target’s model parameters to a variety of source knowledge types. The method relies on pre-posterior distributions, which specifies the distribution of the target’s parameter set given each individual knowledge types. The pre-posteriors are then combined to obtain a posterior distribution for the parameter set that matches all the available knowledge. The method is illustrated with an example.